r - 

TNO-report 

title 

TM-96-B012 

Continuous  haptic  information  and 

TNO  Human  Factors 
Research  Institute 

platform  stabilization  in  target  tracking 

_ j 

•  *  * 


#  #  #  % 


TNO-report 

TM-96-B012 


title 

Continuous  haptic  information  and 
platform  stabilization  in  target  tracking 


TNO  Human  Factors 
Research  Institute 


Kampweg  5 
P.O.  Box  23 
3769  ZG  Soesterberg 
The  Netherlands 

Phone  +31  346  35  62  1 1 
Fax  +31  346  35  39  77 


authors 

J.E.  Korteling 
M.  van  Emmerik 


date 

28  August  1996 


rcnc  GOAL®?  nrerECTED  2 


All  rights  reserved. 

No  part  of  this  publication  may  be 
reproduced  and/or  published  by  print, 
photoprint,  microfilm  or  any  other  means 
without  the  previous  written  consent  of 
TNO. 

In  case  this  report  was  drafted  on  number  of  pages  23  (inch  appendices, 

instructions,  the  rights  and  obligations  of  excl.  distribution  |ist) 

contracting  parties  are  subject  to  either  the 

Standard  Conditions  for  research 

instructions  given  to  TNO,  or  the  relevant 

agreement  concluded  between  the 

contracting  parties. 

Submitting  the  report  for  inspection  to 
parties  who  have  a  direct  interest  is 
permitted. 

®  1996  TNO 


DISTRIBUTION  STATEMENT  A 

Approved  for  public  release; 
Distribution  Unlimited 


TNO  Human  Factors  Research  Institute 
is  part  of  TNO  Defence  research 
to  which  also  belong: 

TNO  Physics  and  Electronics  Laboratory 
TNO  Prins  Maurits  Laboratory 


Netherlands  Organization  for 
Applied  Scientific  Research 


Managementuittreksel 


TNO  Technische  Menskunde,  Soesterberg 


titel 

auteurs 
datum 
opdrachtnr. 
IWP-nr. 
rapport  nr. 


Continuous  haptic  information  and  platform  stabilization  in  target  tracking 
[Continue  haptische  informatie  en  platformstabilisatie  in  een  stuurtaak] 

Dr.  J.E.  Korteling  en  M.  van  Emmerik 

28  augustus  1996 

B96-301 

788.1 

TM-96-B012 


Doel  van  het  onderzoek  was  het  verkrijgen  van  inzicht  in  de  effecten  van  verschillende  vormen 
van  continue  haptische  informatie  (CHI)  op  de  prestaties  van  operators  van  onbemande  platforms 
die  zijn  uitgerust  met  een  camera-monitor  systeem.  In  een  experiment  met  de  TNO-TM  RPV 
simulator  moesten  proefpersonen  een  gesimuleerd  bewegend  (verstoord)  platform  zo  goed  mogelijk 
recht  boven  een  bewegend  doel  houden.  Tijdens  het  uitvoeren  van  deze  besturingstaak  werd  hapti¬ 
sche  informatie  gegeven  over  de  translates  van  het  platform  (en  daarmee  van  de  camera). 

In  het  experiment  werden  twee  besturingsvariabelen  (vier  besturingscondities)  between-subjects 
gemanipuleerd.  Dit  waren:  de  aan-  of  afwezigheid  van  CHI  en  van  (geautomatiseerde)  stabilisatie 
van  het  platform.  Hiervoor  werden  vier  keer  veertien  proefpersonen  gebruikt.  Drie  variabelen 
werden  within-subjects  gevarieerd,  i.e.,  visuele  degradatie  door  vier  verschillende  update 
frequenties  van  het  camerabeeld,  motorisch  taakbelasting  door  hoge  vs.  lage  beweeglijkheid  van 
het  doel  en  visuele  taakbelasting  door  aanwezigheid  of  afwezigheid  van  een  extra  visuele  aandacht- 
taak. 

Proefpersonen  zaten  gedurende  het  experiment  in  een  afgesloten  ruimte  met  een  beeldscherm  voor 
zich.  Hierop  werd  de  visuele  bewegingsinformatie  gegeven.  De  aanvullende  haptische  informatie 
werd  gegenereerd  door  de  uitslagen  van  een  zogenaamde  actieve  joystick.  Hiermee  moest  door 
middel  van  "force  control"  tevens  de  camera  bestuurd  worden. 

Analyse  van  de  stuurfouten  toont  aan  dat  CHI  een  kleinere  volgfout  bewerkstelligt  evenals 
automatische  stabilisatie  van  het  platform.  Deze  effecten  waren  niet  additief.  In  combinatie  waren 
de  effecten  niet  groter  dan  apart.  Terwijl  het  effect  van  platformstabilisatie  afnam  (maar  significant 
bleef)  naarmate  de  update  rate  hoger  werd,  bleken  de  effecten  van  update  rate  en  CHI  additief. 
Wanneer  de  motorische  of  visuele  taakbelasting  toenam,  door  een  beweeglijker  target  of  door  een 
extra  visuele  taak,  steeg  tevens  het  effect  van  platformstabilisatie.  De  grootte  van  het  CHI  effect 
bleek  onafhankelijk  van  taakbelasting. 

Belangrijkste  conclusie:  als  CHI  via  het  besturingsmiddel  betrekking  heeft  op  besturingsinformatie 
leidt  dit  tot  een  betere  stuurprestatie;  als  het  informatie  betreft  die  minder  direct  relevant  is  voor 
besturing,  bijvoorbeeld  ter  verbetering  van  het  situationeel  bewustzijn,  dan  interfereert  het  niet 
substantieel  met  de  stuurprestatie. 
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SUMMARY 

The  present  study  was  conducted  to  gain  insight  in  the  effects  of  different  forms  of  continu¬ 
ous  haptic  information  (CHI)  to  operator  performance  of  an  unmanned  platform  equipped 
with  a  camera-monitor  system.  In  a  simulator  experiment,  subjects  had  to  track  a  moving 
target  keeping  a  simulated  disturbed  platform  as  well  as  possible  straight  above  a  moving 
target.  While  performing  this  tracking  task,  haptic  information  was  provided  concerning 
translations  of  the  platform  (and  thereby  of  camera). 

Two  independent  control  variables  were  manipulated  between-subjects  (four  control 
conditions).  These  factors  were:  presence  or  absence  of  CHI  and  of  (automated)  stabilization 
of  the  platform.  Each  of  the  four  groups  consisted  of  14  subjects.  Three  other  factors  were 
varied  within-subjects,  i.e.,  image  degradation  by  four  different  image  update  rates,  motor 
taskload  through  the  amount  of  target  motion,  and  visual  taskload  by  the  presence  or  absence 
of  an  extra  visual  attention-task. 

During  the  experiment  subjects  were  seated  in  front  of  a  monitor  in  a  closed  room.  This 
screen  was  used  for  visual  information  presentation.  The  supplementary  haptic  information 
was  generated  by  movements  of  a  so-called  active  joystick  which  at  the  same  time  was  used 
for  steering  the  camera  by  “force  control”. 

Analysis  of  the  tracking  errors  shows  that  CHI  as  well  as  automated  platform  stabilization 
substantially  reduced  tracking  error.  These  effects  were  not  additive  as  the  combined  effects 
of  CHI  and  stabilization  were  not  larger  than  the  separate  effects.  Whereas  the  effect  of 
stabilization  diminished  (but  remained  significant)  with  increasing  update  rate,  the  effects  of 
CHI  and  update  rate  were  additive.  With  increasing  motor  or  visual  taskload,  as  a  conse¬ 
quence  of  an  extra  visual  task  or  increased  target  motion,  the  stabilization  effect  increased. 
The  magnitude  of  the  CHI  effect  appeared  independent  of  both  forms  of  taskload. 

Major  conclusion:  CHI  at  the  control  device  improves  tracking  performance  when  it 
involves  information  that  can  be  used  directly  for  vehicle  control;  when  it  provides  informa¬ 
tion  that  is  less  relevant  for  control,  tracking  performance  is  not  substantially  degraded. 
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Continue  haptische  informatie  en  platformstabilisatie  in  een  stuurtaak 

J.E.  Korteling  en  M.  van  Emmerik 

SAMENVATTING 

Om  inzicht  te  krijgen  in  de  eiFecten  van  verschillende  vormen  van  continue  haptische 
informatie  (CHI)  op  de  prestaties  van  operators  van  onbemande  platforms  moesten  proefper- 
sonen  een  gesimuleerd  bewegend  (verstoord)  platform  zo  goed  mogelijk  recht  boven  een 
bewegend  doel  houden.  Tijdens  het  uitvoeren  van  deze  besturingstaak  werd  haptische 
informatie  gegeven  over  de  translates  van  het  platform  (en  daarmee  van  de  camera). 

In  het  experiment  werden  twee  besturingsvariabelen  (vier  besturingscondities)  between- 
subjects  gemanipuleerd.  Dit  waren:  de  aan-  of  afwezigheid  van  CHI  en  van  (geautomati- 
seerde)  stabilisatie  van  het  platform.  Hiervoor  werden  vier  keer  veertien  proefpersonen 
gebruikt.  Drie  variabelen  werden  within-subjects  gevarieerd,  i.e.,  visuele  degradatie  door 
vier  verschillende  update  frequenties  van  het  camerabeeld,  motorische  taakbelasting  door 
hoge  vs.  lage  beweeglijkheid  van  het  doel  en  visuele  taakbelasting  door  aanwezigheid  of 
afwezigheid  van  een  extra  visuele  aandacht-taak. 

Proefpersonen  zaten  gedurende  het  experiment  in  een  afgesloten  ruimte  met  een  beeldscherm 
voor  zich.  Hierop  werd  de  visuele  bewegings informatie  gegeven.  De  aanvullende  haptische 
informatie  werd  gegenereerd  door  de  uitslagen  van  een  zogenaamde  actieve  joystick. 
Hiermee  moest  door  middel  van  “force  control”  tevens  de  camera  bestuurd  worden. 

Analyse  van  de  stuurfouten  toont  aan  dat  CHI  een  kleinere  volgfout  bewerkstelligt  evenals 
automatische  stabilisatie  van  het  platform.  Deze  effecten  waren  niet  additief.  In  combinatie 
waren  de  effecten  niet  groter  dan  apart.  Terwijl  het  effect  van  platformstabilisatie  afnam 
(maar  significant  bleef)  naarmate  de  update  rate  hoger  werd,  bleken  de  effecten  van  update 
rate  en  CHI  additief.  Wanneer  de  motorische  of  visuele  taakbelasting  toenam,  door  een 
beweeglijker  target  of  door  een  extra  visuele  taak,  steeg  tevens  het  effect  van  platform¬ 
stabilisatie.  De  grootte  van  het  CHI  effect  bleek  onafhankelijk  van  taakbelasting. 

Belangrijkste  conclusie:  als  CHI  via  het  besturingsmiddel  betrekking  heeft  op  besturings- 
informatie  leidt  dit  tot  een  betere  stuurprestatie;  als  het  informatie  betreft  die  minder  direct 
relevant  is  voor  besturing,  bijvoorbeeld  ter  verbetering  van  het  situationeel  bewustzijn,  dan 
interfereert  het  niet  substantieel  met  de  stuurprestatie. 
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1  INTRODUCTION 

Over  the  past  decades  the  concept  of  teleoperation  has  evolved  from  fiction  to  reality. 
Developments  in  computer  technology  (miniature  electronics)  and  satellite  navigation 
systems  like  GPS  (Odum,  1995)  combined  with  strong  and  light  materials  from  space 
technology  made  it  possible  to  construct  for  instance  small  unmanned  aerial  vehicles 
(UAV’s)  equipped  to  perform  all  kinds  of  missions  in  environments  potentially  harmful  for 
humans.  But  instead  of  entering  a  Utopia  in  which  subservient  machines  willingly  act  on  our 
wishes  at  the  push  of  a  button,  we  see  ourselves  faced  with  new  jobs,  new  questions  and 
above  all  new  problems.  These  problems  primarily  concern  a  lack  of  coherent  sensory 
information  specifying  the  motions  of  the  platform-camera  system  in  relation  to  the  environ¬ 
ment  and  other  (moving)  objects  in  it. 


1.1  Perceptual  problems  in  platform  control 

During  unmanned  platform  control  the  operators  are  at  a  distant  location  and  thus  lack  direct 
sensory  information.  To  perform  his  basic  tasks  (vehicle  guidance  and  information  acquisi¬ 
tion)  the  operator  depends  on  “substitute  senses”  provided  by  technology.  In  so  far  as 
technology  is  not  (yet)  capable  to  stimulate  all  of  our  senses  accurately,  this  means  practi¬ 
cally  that  only  visual  information  as  supplied  by  a  camera-monitor  system  is  available  to  the 
operator.  Consider  on  the  other  hand  the  sensory  information  available  to  the  pilot  of  a 
manned  aircraft:  "...  [T]he  wide  visual  field  provides  the  pilot  an  immediate  indication  of 
the  slightest  attitude  change  (visual  proprioception).  Small  translational  accelerations  of  the 
aircraft  are  felt  by  the  vestibular  organs.  The  aerodynamic  forces  on  the  control  surfaces  are 
felt  by  the  pressure  and  stretch  receptors  of  the  hand  and  arm.  Speed  can  be  inferred  from 
aerodynamic  noise  or  by  the  sound  of  the  engine.”  (Korteling  &  Van  der  Borg,  1994, 
1997).  The  combination  of  senses  provides  the  pilot  with  a  fairly  unambiguous  picture  of  the 
world  (Patrick,  Sheridan,  Massimino  &  Marcus,  1990).  The  platform  operator  thus  lacks 
this  abundance  of  proprioceptive  information.  This  will  make  his  task  more  difficult  and 
likely  more  demanding. 

Mission-specific  aspects  provide  an  additional  problem.  A  space-platform,  for  example, 
operates  at  such  a  distance  from  its  control  station  that  signal  transmission  suffers  from 
delays  (Stokes,  Wickens  &  Kyte,  1990).  Sometimes  only  limited  bandwidths  can  be  used 
because  the  signal  has  to  be  digitized  and  coded.  This  is  common  practice  in  the  navy  to 
decrease  vulnerability  to  disturbances  and  to  prevent  enemy  jamming  (Korteling  &  Van  der 
Borg,  1994,  1997).  Operating  and  interpreting  the  camera  image  will  be  even  more 
complicated  as  a  consequence  of  restricted  field  of  view  and  low  resolution.  Poor  image 
quality  may  lead  to  difficulties  in  separating  target  motion  and  “self”  motion.  Furthermore, 
when  the  operator  is  unable  to  discriminate  between  movements  of  the  platform  and 
movements  of  the  camera  (as  viewing  direction  and  platform  movements  are  controlled 
independently)  this  will  lead  to  disorientation.  Due  to  the  lack  of  sensory  information  as  well 
as  to  the  problems  just  mentioned,  the  operator  might  experience  serious  problems  with  his 
situation  awareness,  i.e.,  the  perception  of  orientation  and  position  of  the  aircraft  and/or  the 
sensor  in  space  and  time  together  with  an  apprehension  of  the  environment  (threat,  targets), 
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flight  and  system  conditions  (Van  Erp,  Korteling  &  Kappe,  1995).  It  is  hardly  necessary  to 
point  out  that  this  will  be  obstructive  to  mission  performance.  In  addition,  it  has  been 
pointed  out  that  automation  may  reduce  situation  awareness,  which  may  be  an  important 
factor  underlying  accidents  in  complex  man-machine  systems  (e.g.,  Endsley,  1995;  Wickens, 
1992). 


1.2  Continuous  haptic  information 

In  the  early  days  of  aviation,  cables  and  push-rods  directly  linked  the  aircraft’s  control  to  its 
control  surfaces.  The  stiffness  of  the  control  column  then  arises  partly  from  the  aerodynamic 
forces  on  the  rudder.  When  these  forces  are  subject  to  change  (e.g.  as  a  result  of  accelera¬ 
tion)  the  stiffness  of  the  control  column  will  also  change,  by  that  providing  feedback  to  the 
pilot  (Van  Paassen,  1994). 

For  UAV  control  such  a  direct-link  control  system  is  not  feasible.  The  operator’s  commands 
can  no  longer  be  transmitted  mechanically.  Instead  deflections  of  a  joystick  are  translated 
into  electrical  signals  that  are  fed  into  a  flight  control  computer.  The  computer  calculates  the 
desired  rudder  change  that  is,  in  turn,  passed  on  to  the  servo  motors  controlling  rudder 
angle.  Such  a  control  system  is  called  fly  by  wire.  In  these  systems,  the  computer’s  task  is  to 
stabilize  the  aircraft.  The  absence  of  direct  mechanical  links  between  control  and  control 
surface,  however,  allows  for  a  considerable  amount  of  manipulation  of  the  joystick  input. 
This  gave  rise  to  the  development  of  active  steering  devices.  An  active  joystick  consists  of  a 
force  sensor  attached  to  a  joystick.  The  joystick  itself  is  moved  by  motors.  The  motors  get 
their  input  from  a  computer  that  can,  with  the  appropriate  algorithm,  be  coupled  to  any 
variable  of  aircraft  motion.  In  this  way  the  operator  can  be  informed  about  altitude  changes, 
climb  speed,  direction  changes  of  the  aircraft  etc.  through  the  position  of  the  stick  (Korteling 
&  Van  Gent,  1994;  Van  Paassen,  1994).  The  success  of  an  active  stick  will  depend  on  a 
proper  choice  of  the  feedback  variable  and  the  manipulation  of  joystick  input. 

With  the  availability  of  advanced  computer  systems  an  obvious  next  step  in  UAV-control 
would  be  to  automatize  the  task  completely.  This  is  not  without  problems  however. 
According  to  Wickens  (1992)  the  concept  of  flexibility  is  of  great  importance.  There  seems 
to  be  a  trade  off  between  the  range  of  capabilities  and  the  degree  of  flexibility  of  a  man- 
machine  system  (Korteling  &  Van  der  Borg,  1994,  1997).  Automation  requires  that  a 
mission  is  planned  and  programmed  a  priori.  Tasks  like  controlling  a  UAV  are  sometimes 
difficult  to  plan  in  advance  due  to  pursuit  tracking  aspects.  They  require  a  human  operator 
to  obtain  the  necessary  level  of  flexibility.  It  seems  then,  the  optimal  solution  is  partial 
automation  of  platform-control.  Although  this  will  relieve  the  operator’s  taskload,  it  places 
the  operator  partially  outside  the  control  loop  which  is  considered  a  potential  drawback  of 
automation.  Without  precise  knowledge  of  “what  the  system  is  doing,  why  it  is  doing  it  and 
what  it  will  do  next”,  a  loss  of  situation  awareness  may  be  the  result  (Wickens,  1992). 

Application  of  CHI  may  enhance  situation  awareness  of  remote  operators.  The  potential 
benefit  of  haptic  information  has  also  been  demonstrated  in  other  fields  such  as  car  driving 
and  aircraft  control.  In  a  study  on  car  driving,  continuous  force  feedback  concerning  the 
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lateral  position  of  the  car  on  the  road  was  provided  through  an  active  steering  wheel.  In  this 
experiment  a  reduction  of  both  control  error  (deviation  of  lateral  position)  and  control  effort 
(driver’s  steering  wheel  movements)  was  observed  (Schumann,  Lowenau  &  Naab,  1994).  In 
an  exploratory  simulator  study,  haptic  feedback  through  an  active  gas-pedal  was  found  to  be 
of  help  in  maintaining  a  constant  speed,  thereby  reducing  taskload  (Schumann,  Godthelp  & 
Hoekstra,  1992). 

An  active  stick  was  also  found  to  be  an  aid  in  aircraft  control  (Laurman  &  Verhoeven, 
1993).  In  a  simulator  study,  subjects  had  to  approach  a  runway.  Subjective  taskload  ratings 
(Cooper-Harper  scale)  and  standard  deviations  from  glide  path  and  speed  during  this 
approach  were  used  as  an  index  of  task  performance  and  compared  for  different  controls. 
The  active  stick  informed  subjects  of  either  roll  speed  and  heading  angle  velocity  or  roll 
speed  and  vertical  disturbance  speed.  Although  both  subjective  ratings  and  standard 
deviations  indicated  superior  performance  with  the  active  controls  compared  to  a  passive 
stick,  methodological  limits  of  the  experiment  made  it  impossible  to  reliably  differentiate 
between  the  specific  feedback  algorithms. 

Using  auditory  and  force  feedback,  Massimino  and  Sheridan  (1993,  1994)  tested  subjects 
with  several  peg-in-hole  telemanipulation  tasks.  In  these  tasks,  subjects  had  to  put  a  peg  into 
holes  of  different  size  using  a  remotely  operated  robot  arm  which  could  be  controlled  in  7 
degrees  of  freedom.  The  authors  concluded  that  force  feedback  could  compensate  for 
conditions  of  degraded  visual  information.  The  effect  of  force  feedback  on  task  time  at  an 
image  update  rate  of  3Hz  was  found  to  be  of  the  same  magnitude  as  an  increase  in  update 
rate  of  the  visual  information  from  3  to  30Hz  without  force  feedback.  For  better  visual 
conditions  (i.e.  higher  update  frequencies)  the  effect  was  less  pronounced. 

Results  in  a  study  by  Korteling  and  Van  der  Borg  (1994,  1997)  are  in  agreement  with  these 
findings.  In  their  study  they  had  subjects  to  operate  a  semi-autonomous  interface  for  a 
simulated  UAV.  The  system  automatically  compensated  for  translations  of  the  platform 
relative  to  the  earth  and  accompanied  these  system  interventions  by  the  appropriate  joystick 
movements.  As  a  result  subjects  received  tactile  (haptic)  information  of  the  compensatory 
interventions  of  the  system.  The  magnitude  of  the  active-interface  effect  was  equal  to  the 
effect  of  an  increase  in  update-frequency  from  2  to  5Hz. 

A  study  concerning  helicopter  control  also  found  a  beneficial  effect  of  haptic  information 
(Korteling  &  Van  Gent,  1994).  The  effect  was  limited,  however,  to  altitude  maintenance  and 
appeared  most  pronounced  in  high  taskload  conditions.  It  is  therefore  suggested  that  haptic 
information  might  particularly  be  of  help  to  compensatory  control  tasks  in  demanding 
situations  (Kelley,  1968;  Korteling  &  Van  Gent,  1994). 


1.3  The  present  experiment 

The  present  study  was  undertaken  to  investigate  the  effects  of  continuous  haptic  information 
(CHI)  in  a  tracking  task.  It  was  investigated  if  CHI  could  contribute  to  reducing  operator 
taskload  and  would  facilitate  operator  performance.  It  was  also  considered  of  interest  to 


8 


evaluate  the  usefulness  of  CHI  in  conditions  of  high  automation.  Under  these  very  circum¬ 
stances,  information  on  the  nature  of  the  system’s  actions  may  help  to  keep  the  operator  in 
the  control  loop,  i.e.,  enhancing  his  situation  awareness  (e.g.  Korteling  &  Van  der  Borg, 
1994,  1997;  Wickens,  1992). 

In  the  present  experiment  subjects  had  to  track  the  target  in  a  task  situation  that,  in  an 
abstract  way,  represented  the  tracking  of  a  moving  target  from  a  moving  platform  with  a 
camera-monitor  system.  Subjects  used  either  an  active  joystick  providing  CHI  or  a  passive 
joystick.  These  joystick  types  were  combined  with  two  levels  of  stabilization  (either  subjects 
had  to  stabilize  the  platform  while  tracking  the  moving  object,  or  the  stabilizing  task  was 
done  automatically  by  the  connected  computer  systems).  This  resulted  in  four  task  conditions 
(or  stick  modes). 

By  manipulating  the  update  frequency  of  the  monitor  image,  perceptual  degradation  was 
varied.  Motor  load  was  controlled  by  varying  target  mobility.  It  was  investigated  in  which 
way  these  factors  affected  task  performance  in  interaction  with  CHI  and  automation. 
Furthermore,  a  secondary  task  was  developed.  This  task  forced  subjects  to  divide  visual 
attention,  however,  without  causing  structural  interference  (by  visual  scanning  require¬ 
ments).  Decrease  in  tracking  task  performance  due  to  extra  visual  attention  requirements 
(visual  load)  was  expected  to  be  counteracted  by  availability  of  CHI. 

Because  platform  control  is  complicated  by  a  number  of  independent  control  variables,  (e.g. 
camera  angle,  viewing  direction  and  vehicle  heading)  in  this  study  a  simplified  2D  paradigm 
of  tracking,  without  stereoscopic-  or  perspective  information  was  favoured  over  a  more 
complex  3D  paradigm.  By  assuming  the  camera  to  be  aimed  perpendicular  to  the  earth  no 
perspective  or  other  depth  cues  were  involved.  With  the  camera  angle  fixed,  platform 
control  and  camera  control  were  completely  dependent.  It  was  expected  that  this  simplifica¬ 
tion  would  allow  for  measurement  of  the  effects  of  CHI  on  platform  control  with  a  minimum 
of  interference  from  other  (complicating)  factors. 

One  potential  drawback  of  the  2D  image  is  a  reduced  correspondence  with  the  real  world 
situation.  This  is  however  regarded  of  minor  importance  as  the  main  objective  was  not  to 
simulate  a  real  operator  room.  The  study  was  primarily  focused  on  the  possibility  of  using 
haptic  information  as  a  supplement  to  visual  information,  in  an  object- tracking  including 
disturbance  motions  of  the  (own)  platform. 

With  regard  to  the  results,  clear  effects  of  image  update  frequency,  target  mobility  and 
visual  attentional  load  were  expected.  More  interesting,  however,  were  the  differences 
between  the  stick  configurations  and  their  interactions.  It  was  expected  that  tracking  error 
would  become  significantly  smaller  with  the  active  stick  configurations  compared  to  their 
passive  counterparts.  In  related  research  the  benefits  of  haptic  information  were  most 
pronounced  under  extreme  and  difficult  circumstances.  Therefore,  it  was  expected  that  the 
aforementioned  advantage  would  decrease  in  the  configurations  with  automatic  platform- 
stabilization.  The  stick-mode  effects  were  expected  to  increase  again  with  conditions  of  low 
image  update  rate,  increased  motor-  and  visual  load. 
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Furthermore,  if  the  results  would  indicate  minimal  negative  effects  of  CHI  with  automatic 
platform  stabilization,  the  idea  of  enhancing  situation  awareness  by  providing  CHI  in  semi- 
automated  control  tasks  will  be  a  promising  opportunity  requiring  further  investigation. 


2  METHOD 
2.1  Instrumentation 

The  experiment  was  conducted  in  the  TNO-TM  Teleoperation  Research  Simulator.  This 
facility  is  especially  designed  for  simulating  teleoperation  missions,  such  as  with  remotely 
piloted  vehicles  (Korteling  &  Van  Breda,  1994).  Detailed  specifications  are  given  in 
Appendix  A.  In  the  present  experiment,  the  chair  was  placed  at  a  marked  spot  to  make  sure 
subjects  kept  a  roughly  equivalent  distance  from  the  monitor.  The  subjects  had  a  joystick  to 
their  right  and  a  push-button  to  their  left.  Straight  in  front  of  them,  at  a  distance  of  approxi¬ 
mately  80cm,  a  19in  RGB  high-resolution  monitor  (Mitsubishi  colour  display  monitor)  was 
placed.  The  foot  pedals  of  the  mock-up  were  temporarily  removed. 

Joystick  control  was  achieved  by  two  independent  servomechanisms,  which  received  their 
input  from  a  force  sensor.  Maximum  stick  deflection  range  was  +/—  30°.  Only  one-third  of 
the  total  stick  range  could  be  used  for  control  (simulation  of  stick  suspension)  because  the 
remaining  two-third  had  to  be  reserved  for  stick-motion  information  in  the  active-stick 
conditions.  To  obtain  this  one-third  of  total  stick  range  a  force  of  2  N  had  to  be  applied  to 
the  stick.  Maximal  speed  of  compensation  for  disturbance  error  was  7.95cm/s.  The  relation 
between  joystick  force  (N)  and  screen  speed  was  linear.  The  joystick  extended  9cm  above  its 
pivot.  A  486  PC  (IBM  compatible)  was  used  to  compute  the  changes  in  target  position 
caused  by  force  input  on  the  stick.  This  information  was  passed  on  to  a  Silicon  Graphics 
computer  (IRIS  4D)  and  was  used  to  generate  the  monitor  image.  Another  486  PC  (IBM 
compatible)  was  used  for  scenario  generation  and  was  responsible  for  data  storage  (10Hz 
sampling  frequency). 


2.2  Subjects 

The  experiment  was  conducted  with  four  groups  of  14  right-handed  male  subjects  with  high 
educational  level  (VWO/HBO  or  university).  Mean  age  of  the  subjects  was  22  years  and  11 
months  (range  17-30,  SD  2.48).  All  subjects  had  normal  or  corrected-to-normal  vision. 
Subjects  were  matched  for  “information  processing  speed”  (Salthouse,  1985)  as  measured  by 
the  WAIS-subtest  “Digit-Symbol  Substitution”  (Wechsler,  1981).  The  subjects  were  paid  for 
their  participation. 
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2.3  The  experimental  tasks 

Subjects  were  seated  at  a  distance  of  approximately  80cm  from  the  monitor.  They  had  to 
track  a  target  in  a  task  situation  that,  in  an  abstract  way,  represented  the  tracking  of  a 
moving  target  from  a  moving  platform  with  a  camera-monitor  system.  The  image  on  the 
screen  represented  an  image  of  a  flat  background  (environment)  with  one  target),  as  seen  by 
a  camera  fixed  on  a  (moving)  platform.  Platform  motions  were  parallel  to  the  flat  back¬ 
ground.  The  camera  was  aimed  perpendicular  to  the  environment.  As  a  result  of  this 
viewing-angle  the  image  represented  a  2D  picture.  The  target  and  the  camera  on  the 
platform  only  translated  in  the  x-  and  y-  axis.  There  were  no  rotations.  The  background  was 
created  by  a  hundred  plus  signs,  randomly  distributed  over  a  black  screen.  A  white  cross¬ 
hair  (0,4°),  representing  the  visor  of  the  camera,  was  displayed  in  the  centre  of  the  screen. 
Any  joystick  movement  resulted  in  a  corresponding  displacement  of  the  background  relative 
to  the  cross-hair.  During  the  experiment,  the  target  (a  small  diamond,  0,4°)  would  move 
across  the  background.  Subjects  were  instmcted  to  control  the  joystick  with  their  right  hand 
thumb  and  index  finger  and  to  keep  the  visor  aimed  at  the  target  as  well  as  possible.  The 
distance  between  target  and  visor  during  tracking  was  continuously  measured  in  order  to 
indicate  performance. 

To  simulate  the  disturbing  forces  of  wind  gusts  up  in  the  air,  the  camera  platform  and  the 
target  were  disturbed  by  a  disturbance  function  using  different  starting  points  within  this 
function  for  both  elements.  This  function  caused  the  platform/camera  as  well  as  the  target  to 
change  direction  randomly  at  a  rate  of  30Hz  while  they  moved  at  a  constant  speed  of  5.30 
or  2. 65cm/ s  over  the  screen,  respectively.  The  unfiltered  direction  changes  were  limited  to  a 
maximum  of  10°  or  20°,  depending  on  the  motor  taskload  condition.  The  direction  changes 
were  filtered  such  that  the  frequency  of  direction  changes  could  not  exceed  0.625  or  1.25Hz, 
depending  on  the  motor  load  condition. 

For  the  dual  task,  eight  small  rectangles  were  placed  in  a  circle  at  a  distance  of  4cm  around 
the  visor.  These  were  always  displayed  to  keep  the  number  of  items  on  the  screen  constant. 
As  explained  below,  the  rectangles  were  used  in  manipulating  the  amount  of  visual  taskload. 


2.4  Independent  variables 

In  the  experiment  six  factors  were  varied,  two  between  subjects  (platform  stabilization  and 
CHI),  which  resulted  in  the  four  tracking  modes,  and  four  within  subjects  (update 
frequency,  motor  load,  visual  load  and  replication). 

Automated  platform  stabilization 

The  platform/camera  could  be  disturbed  as  described  above.  In  the  automated  stabilization 
conditions  this  disturbance  was  completely  removed  such  that  the  platform  remained  stable. 
Consequently,  in  these  conditions  camera  motions  were  completely  generated  by  subject- 
initiated  forces  on  the  joystick. 
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CHI 

The  platform/camera  could  be  controlled  by  either  a  passive  joystick  or  an  active  joystick 
providing  CHI  by  its  motions.  The  CHI  involved  information  concerning  either  p  a  form 
Liability  or  the  system-initiated  compensatory  actions  aimed  at  nulling  the 5  plat  om 
instability.  The  amplitude  of  the  active  stick  movements  constantly  took  up  one  third  of  the 
total  stick  range  because  platform  speed  was  also  constant.  The  largest  possible  steering 
input  subjects  had  to  give,  never  exceeded  two  third  of  the  stick  range  so  that  presence  of 
CHI  did  not  limit  target  control. 

The  aim  of  the  information  transferred  by  the  stick  movements  depended  on  the  stick 
condition.  While  controlling  an  instable  platform,  CHI  informed  the  operator  on  how  to 
correct  instabilities  himself.  The  position  of  the  stick  indicated  the  direction  of  Platfo™ 
disturbance  which  could  be  corrected  by  steering  in  the  opposite  direction.  When  the 
platform  was  stabilized,  CHI  informed  the  operator  on  the  correcting  actions  of  the  system. 
Stick  position  indicated  what  the  system  was  doing  to  stabilize  the  platform.  This  knowledge 
of  the  system’s  actions  would  help  the  operator  to  maintain  a  complete  representation  of  all 
flight  aspects  (keeping  the  man  in  the  loop)  thereby  enhancing  situation  awareness. 


Tracking  modes 

The  combination  of  platform  stabilization  and  CHI  resulted  in  the  following  four  tracking 
modes. 

Mode  0  was  obtained  by  combining  a  passive  joystick  with  an  instable  camera  platform. 
Target  tracking  movements  had  to  be  superimposed  on  the  steering  input  required  for 
platform  control . 

Mode  1  involved  the  same  passive  control  stick.  The  platform,  however,  was  stabilized 
automatically  and  would  not  move  unless  the  stick  was  moved. 

Mode  2  was  similar  to  mode  0  except  for  the  fact  that  subjects  used  an  active  joystick  to 
control  the  platform.  The  active  joystick  provided  the  subjects  with  CHI  by  its  tilt.  By 
counteracting  on  the  computer-generated  joystick  movements  the  instability  of  the  platform 
had  to  be  compensated  for.  On  top  of  these  actions,  tracking  movements  had  to  be  super¬ 
imposed.  (In  this  condition  CHI  was  comparable  to  the  information  used  by  Korteling  & 

Van  Gent,  1994.) 

Mode  3  combined  an  active  joystick  with  automatic  stabilization.  The  function  of  the  active 
stick  was  to  give  CHI  on  the  compensatory  actions  made  by  the  computer.  Tracking 
movements  thus  had  to  be  superimposed  on  the  computer-generated  stick  movements.  (CHI 
in  this  mode  was  comparable  with  the  haptic  information  previously  used  by  Korteling  & 
Van  der  Borg,  1994,  1997.)  In  fact,  this  condition  involved  a  pursuit  target  tracking  task 
with  system-initiated  joystick  motions  that  may  be  expected  to  produce  interference  with  the 
tracking  task. 


Update  frequency 

Based  on  theoretical  and  practical  criteria,  update  frequency  of  the  monitor  image  was  either 
2  3,  5  or  30Hz.  With  update  frequencies  lower  than  5Hz  the  perception  of  movement  of  the 
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target  gradually  deteriorates  until  it  disappears  completely  at  2Hz  and  down.  It  will  cause 
target  motion  to  become  unpredictable  to  the  operator  (this  also  depends  on  the  magnitude  of 
the  target  movements).  With  an  update  frequency  of  30Hz,  on  the  other  hand,  complete 
image  fusion  is  provided.  Therefore,  it  is  regarded  as  desirable  (Massimino  &  Sheridan, 
1994).  At  the  same  time,  some  UAV  systems  currently  used  in  military  settings  are  not  yet 
able  to  reach  update  frequencies  above  5Hz  (Van  Breda  &  Passenier,  1993). 

Motor  load 

The  amount  of  motor  taskload  was  manipulated  by  varying  the  range  of  direction  changes 
generated  by  the  random  disturbance  signal.  This  counted  for  the  disturbance  of  the  camera 
platform  as  well  as  for  the  target.  Direction  changes  by  this  signal  were  limited  to  a 
maximum  of  10°  (low  load)  or  20°  (high  load)  to  the  left  or  right  and  were  generated 
randomly  at  30Hz.load  condition.  The  direction  changes  were  filtered  such  that  the  fre¬ 
quency  of  direction  changes  were  limited  to  0.625Hz  in  the  low-load  condition  or  1.25Hz  in 
the  high-load  condition. 

Visual  load 

The  amount  of  visual  taskload  was  varied  by  using  an  extra  visual  attention  task  with 
minimal  motor  demands  (Fig.  1).  The  task  was  also  designed  such  that  it  could  not  generate 
structural  visual  interference,  i.e.,  by  provoking  eye  movements  from  the  target.  This  task 
forced  subjects  to  spread  their  visual  attention  from  the  visor  in  the  centre  of  the  screen  to 
an  area  of  about  6°  in  diameter  around  the  visor.  Along  the  edge  of  this  imaginary  circle, 
eight  small  blocks  (0.7°  diagonal)  were  positioned.  The  circle  rotated  slowly  (1.25°/s)  to 
prevent  fading  of  the  blocks  as  a  consequence  of  too  strong  eye-fixations. 

During  a  high  visual  taskload  trial,  two  of  the  blocks  now  and  then  disappeared  from  view 
for  a  time  of  100ms.  Subjects  had  to  judge  the  smallest  area  between  the  disappearing  blocks 
and  respond  only  when  two  blocks  were  in  between  (this  number  of  remaining  blocks  could 
be  0,  1,  2,  or  3).  This  was  the  case  in  twelve  out  of  twenty-four  stimuli  per  trial.  Responses 
had  to  be  made  by  pushing  a  button  with  the  left  (i.e.,  non-dominant)  index-finger.  If  no 
response  was  initiated  within  1. 5s,  a  tone  was  presented  until  the  button  was  pressed.  In  case 
of  a  false  alarm,  two  short  beeps  followed  the  response.  Subjects  had  to  keep  the  number  of 
errors  and  late  reactions  below  5  % .  In  order  to  keep  motor  demands  as  low  as  possible  this 
task  just  required  subjects  to  react  accurately.  Hence,  apart  from  the  time  limit,  reaction 
time  was  not  relevant. 

Replication 

After  the  presentation  of  all  16  within-subjects  conditions,  in  the  first  16  trials,  all  these 
trials  were  presented  once  more  and,  per  subject,  in  the  same  order  to  check  possible  effects 
due  to  learning  or  fatigue. 


+  + 


Fig.  1  The  eight  blocks  around  the  visor,  used  for  the  extra  visual  attention 
task,  displayed  on  the  background  consisting  of  randomly  located  +  signs. 

2.5  Dependent  variables  and  data  analysis 

In  order  to  measure  tracking  performance,  Root  Mean  Squared  (RMS)  tracking  error  was 
computed  for  each  trial.  Error  was  defined  as  the  deviation  of  the  centre  of  the  tracking 
symbol  on  the  monitor  screen  from  the  centre  of  the  target  in  pixels  on  the  screen.  This 
deviation  was  sampled  by  10Hz.  As  the  first  four  seconds  of  each  trial  were  not  used,  RMS 
was  based  on  a  total  number  of  1 16s  xlOHz=  1160  deviation-scores  per  trial.  On  these  RMS 
values  analyses  of  variance  (ANOVA)  were  performed.  A  2(platform-stabilization)  x 
2(CHI)  between  groups  design  was  used  with  four  repeated  measure  factors:  image  update 
rate  (4  levels)  x  motor  load  (2)  x  visual  load  (2)  x  replication  (2).  The  number  of  errors 
(incorrect  responses  or  misses)  in  the  dual-task  conditions  was  far  below  the  5%  margin. 
These  data  will  therefore  not  be  further  discussed. 
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2.6  Procedure 

The  experiment  plus  training  lasted  one  morning  or  afternoon  (3  hours)  during  which  two 
subjects  were  tested.  At  any  time,  only  one  subject  was  performing  the  task  while  the  other 
subject  rested  in  the  adjacent  room. 

Practice  session 

Before  running  practice  trials,  a  briefing,  involving  the  reading  of  a  brief  instruction,  was 
done  as  to  the  nature  of  the  experiment  and  the  tasks  to  be  performed  (see  Appendix  B). 
Subsequently,  the  subjects  practised  together  for  about  40  minutes  (8  trials  of  each  120s  for 
each  subject).  In  a  pilot  experiment  this  was  found  to  be  sufficient  to  achieve  stable 
performance  for  each  stick-mode.  During  the  practice  session  the  experimentator  answered 
possible  questions  related  to  the  task. 

Experimental  trials 

In  the  experiment,  32  trials  were  presented  to  the  subjects,  i.e.,  image  update  rate  (4  levels), 
visual  load  (2),  motor  load  (2),  replication  (2).  These  were  divided  in  four  eight-trial  blocks. 
For  each  subject,  all  trials  were  performed  in  the  same  stick  mode  (one  out  of  the  four 
modes)  throughout  the  experiment.  Trial  duration  was  120s.  Between  trials  subjects  paused 
for  20s.  After  completing  each  block  of  trials,  the  subjects  traded  places.  The  order  of 
conditions  was  balanced  between  subjects. 


3  RESULTS 

3.1  Automated  stabilization  and  CHI 

A  main  effect  of  automation  showed  that  tracking  performance  with  an  automatically 
stabilized  platform  was  superior  relative  to  an  instable  platform  [F(l,52)=79.279, 
p<  <  .01  j.  The  passive  stick  clearly  benefitted  from  automated  platform-stabilization.  For 
the  active  stick,  this  stabilization  benefit  just  failed  to  reach  significance  (Fig.  2). 

In  addition,  a  significant  main  effect  for  CHI  was  found  indicating  that  subjects  performed 
better  in  the  active  joystick  modes  involving  CHI  [F(l,52)=5.738,  p<  .05].  Although 
subjects  performed  better  with  CHI,  this  effect  turned  out  to  be  limited  to  the  conditions 
without  automated  platform-stabilization.  When  the  platform  was  stabilized  automatically,  no 
significant  differences  were  found  between  the  active  and  passive  stick  according  to  a  post- 
hoc  analysis  (Tukey  HSD  test)!  Stabilization  and  CHI  were  thus  not  additive.  Fig.  2  shows 
this  interaction  [F(  1,52) =37. 473,  p<<  .01],  Only  without  stabilization,  CHI  provided 
better  tracking  performance. 
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Fig.  2  Tracking  error  as  affected  by  CHI  and  platform  stabilization. 


3.2  Other  independent  variables 

The  data  also  showed  a  significant  main  effect  of  image  update  rate  [F(3,156)  =  355.95, 
p<  <  .01],  Tracking  performance  improved  with  increasing  update  frequency  of  the  camera 
image.  In  Fig.  3,  RMS  error  is  displayed  for  each  of  the  four  different  image  update  rates. 
There  was  a  significant  motor-load  effect  indicating  an  increment  in  RMS  error  with 
increasing  target  motion  [F( 1,52) =562.526,  p<  C.01].  When  subjects  had  to  perform  the 
extra  visual  attention  task,  tracking  performance  also  deteriorated  significantly  [F(l,52)  = 
18.259,  p<  <0.01]. 

No  effect  of  replication  was  found,  showing  that  performance  did  not  increase  (training 
effect)  or  decrease  (fatigue  effect)  during  the  experiment. 


Fig.  3  Mean  RMS  tracking  error  as  affected  by  update  frequency. 
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3.3  Interactions 

Neither  visual  taskload  nor  update  frequency  or  tracking  difficulty  interacted  with  CHI. 
There  was,  however,  a  significant  interaction  effect  of  automation  x  update  rate  [F(3,156) 
=25.983,  p<  C.01].  Although  the  effect  remained  significant,  there  was  a  decrease  in  the 
benefit  of  automation  with  increasing  update  frequency  (Fig.  4a).  Automatic  stabilization 
also  interacted  significantly  with  motor  load  [F(l, 52)  =  132.967,  p<  <  .01],  Tracking 
performance  was  degraded  most  by  increasing  target  motion  when  the  platform  was 
unstabilized  (Fig.  4b). 


Fig.  4  Mean  tracking  performance  with  a  stabilized  and  a  non-stabilized 
platform  as  affected  by  update  frequency  (a)  and  motor  taskload  (b). 


Fig.  5  Mean  tracking  performance  with  various  update  frequencies  as  affected 
by  visual  taskload  (a)  and  motor  taskload  (b). 
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Furthermore,  update  rate  interacted  with  visual  load  as  manipulated  by  the  visual  attention 
task  [F(3,156)  =  9.615,  p<  C.01].  The  difference  in  RMS  tracking  error  for  conditions  of 
absence  or  presence  of  the  visual  attention  task  disappeared  at  a  frequency  of  30Hz.  Below 
this  value,  it  was  significant  and  of  constant  magnitude  (Fig.  5a).  A  similar  effect  was 
observed  for  the  interaction  of  update  frequency  x  motor  load.  At  30Hz  the  (significant) 
difference  between  the  two  levels  of  motor  load  was  smallest  (Fig.  5b). 


4  DISCUSSION 

In  everyday  life  people  are  hardly  aware  of  their  dependence  on  haptic  information.  It  is  not 
until  other  sensory  information  is  degraded  or  cannot  be  sampled  that  touch  is  fully 
appreciated  (e.g.  finding  a  keyhole  in  the  dark  or  when  reaching  under  one’s  seat  while 
driving  a  car).  In  remote  control  settings,  the  operator  usually  lacks  haptic  information  while 
visual  information  is  restricted  and  distorted  as  a  consequence  of  a  limited  viewing  angle  and 
low  image  update  frequency  and/or  resolution.  At  present,  there  is  still  too  little  knowledge 
regarding  the  possibilities  and  limitations  of  haptic  information  and  the  effects  of  interacting 
factors  in  order  to  formulate  design  principles  for  the  increasing  number  of  remote  control 
tasks.  Therefore,  research  on  the  possibilities  and  limitations  of  presentation  of  haptic 
information  in  teleoperation  should  be  conducted  in  order  to  contribute  to  this  knowledge. 
This  simulator  study  was  conducted  to  provide  more  insight  in  these  complicating  factors  in 
platform  control,  in  particular  in  haptic  information  provided  by  control  devices. 

In  the  present  experiment  haptic  information  was  provided  through  joystick  movements.  The 
operator  controlled  a  moving  platform-with-camera  to  track  a  moving  target.  The  platform 
was  considered  instable  as  a  consequence  of  various  unpredictable  forces  such  as  wind  gusts. 
Continuous  haptic  information  (CHI)  consisted  of  stick  movements  that  informed  the 
operator  on  the  direction  of  these  forces.  In  one  mode,  these  forces  provided  information 
concerning  the  joystick  forces  that  had  to  be  applied  in  order  to  compensate  for  platform 
disturbances.  In  this  case,  platform  stabilization  was  acquired  just  by  counteracting  as  well 
as  possible  the  system-initiated  joystick  motions.  In  one  other  mode,  CHI  did  not  provide 
such  information  because  the  platform  was  already  stabilized.  This  condition  was  incorpo¬ 
rated  in  order  to  test  the  negative  effects  of  the  system-initiated  joystick  motions.  However, 
in  practical  task  settings,  this  kind  of  CHI  can  be  beneficial  as  providing  additional  informa¬ 
tion  (situation  awareness)  concerning  system  automation.  Automation  may  degrade  situation 
awareness  and  therefore  is  considered  a  significant  factor  underlying  human  error  and 
accidents  (Endsley,  1995;  Wickens,  1992).  In  the  present  experiment  automation  consisted 
of  (perfect)  system  ’corrections’  for  the  external  forces  disturbing  the  platform  on  which  the 
camera  was  mounted,  e.g.,  gyroscopic  stabilization.  This  was  called  automatic  stabilization. 
For  the  present  experiment,  it  was  not  expected  that  this  haptic  information  for  increasing 
situation  awareness  would  enhance  performance  because  it  was  not  relevant  for  the  tracking 
task.  Actually,  this  condition  involved  a  pursuit  target  tracking  task  entailing  system-initiated 
joystick  motions  that  may  be  expected  to  interfere  with  the  tracking  task.  The  question  here 
involved  the  degree  to  which  this  continuous  haptic  interference  would  hamper  tracking 
performance  as  such.  If  the  results  would  indicate  minimal  negative  effects  of  joystick 
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motion  with  automatic  platform  stabilization,  the  idea  of  enhancing  situation  awareness  by 
CHI  in  semi-automated  control  tasks  would  be  a  promising  opportunity. 

Compared  to  tracking  performance  in  the  passive  joystick  conditions,  in  which  no  automatic 
stabilization  or  CHI  was  provided,  the  data  showed  that  subjects  performed  substantially 
better  with  both  kinds  of  support.  The  beneficial  effect  of  automation  was  most  pronounced, 
followed  by  the  combined  effects  of  automation  and  CHI,  CHI  only,  and  finally  no  support 
at  all.  It  is  clear  from  these  results  that  both  haptic  information  and  (of  course)  automation 
are  of  help.  The  effects,  however,  clearly  are  not  additive  as  the  combined  effects  of  CHI 
and  stabilization  were  not  larger  than  the  separate  effects. 

To  explain  the  results  a  closer  look  at  the  task  dynamics  will  be  helpful.  The  task  at  hand 
involved  in  fact  both  a  pursuit  and  a  compensatory  component  (tracking  the  target  and 
correcting  platform  instability,  respectively).  When  the  system  takes  care  of  stabilization,  the 
operator  is  only  responsible  for  the  pursuit  tracking  component.  Therefore  performance 
improves  substantially  with  automatic  stabilization.  The  fact  that  addition  of  CHI  in  this 
condition  led  to  a  small  (insignificant)  decrease  in  tracking  performance  can  be  explained  on 
the  basis  of  a  point  made  by  Kelley  (1968).  He  stressed  that  continuous  active  signals  disturb 
the  operator’s  internal  model  of  the  steering  dynamics  and  thereby  will  interfere  with  the 
operation  of  a  mechanical  device.  However,  because  exclusive  CHI  led  to  clear  performance 
improvement  in  the  non-stabilized  condition,  it  can  be  concluded  that  the  interfering  effects 
of  CHI  on  the  pursuit  component  apparently  are  smaller  than  the  positive  effects  of  provid¬ 
ing  information  that  can  be  used  for  compensating  platform  disturbances.  Therefore,  the 
present  data  indicate  that  the  interference  by  stick  motion,  as  described  by  Kelley  (1968), 
was  relatively  small  and  may  be  cancelled  when  positive  effects  of  the  active  signals  (e.g. 
aimed  at  enhancing  situation  awareness)  are  sufficiently  large.  On  the  basis  of  a  more 
tentative  study,  this  already  was  envisioned  by  Korteling  and  Van  der  Borg  (1994,  1997). 
Hence,  the  addition  of  CHI  in  case  of  automatic  stabilization  of  platform  disturbances  may 
enhance  situation  awareness,  whereas  it  slightly  decreases  tracking  performance  per  se.  This 
trade-off  has  to  be  taken  into  consideration  for  each  specific  man-machine  system  that  uses 
automatic  stabilization.  For  example,  an  operator  of  an  unmanned  aircraft  may  benefit  from 
feeling  that  the  airframe  is  turning  while  his  outside  image  remains  earth-fixed. 

A  pronounced  main  effect  of  update  frequency  was  found.  The  effect  of  an  increase  in 
update  rate  (from  2Hz  to  5Hz  or  from  3Hz  to  30Hz)  was  of  the  same  magnitude  as  the 
effect  of  CHI  whereas  an  increase  (from  2Hz  to  30Hz)  equalled  the  effect  of  automatic 
stabilization. 

Negative  performance  effects  of  extra  visual-  or  motor  taskload  and  positive  effects  of 
automatic  stabilization  decreased  with  increasing  update  frequency.  For  the  extra  visual 
attention  task,  this  interaction  can  be  explained  by  the  fact  that  with  low  update  rates,  the 
task  becomes  more  demanding  such  that  less  spare-capacity  is  left  for  extra  tasks,  in  this 
case  the  visual-spatial  attention  task,  or  task-complications  that  depend  on  the  same  visuo- 
spatial-manual  functions  or  resources  (i.e.,  Wickens,  1987,  1992).  The  decreasing  effects  of 
disturbance  motion  and  automatic  stabilization  with  update  frequency  can  be  explained  by 
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the  fact  that  low  frequencies  reduce  the  amount  of  visual  motion  information,  which  will  be 
more  detrimental  as  the  amount  of  motion  to  be  compensated  increases. 

The  effect  of  automation  was  found  to  be  highly  advantageous  to  tracking  performance  as  a 
consequence  of  eliminating  the  need  for  compensatory  control  by  the  operator.  In  other 
words:  having  to  compensate  disturbances  while  tracking  a  moving  target  makes  the  tracking 
task  more  difficult.  As  the  effect  of  an  increase  in  platform  disturbances  and  target  movabil- 
ity  affected  both  pursuit  and  compensatory  tracking  performance,  it  was  observed  not 
surprisingly,  that  increased  disturbances  were  less  detrimental  to  tracking  performance  in 
conditions  of  automated  platform  stabilization.  Because  the  increase  in  compensatory 
tracking  effort  was  dealt  with  by  the  system,  the  only  component  suffering  was  pursuit 
tracking.  For  the  extra  visual  attention  task  no  such  interaction  with  automation  was  found, 
that  is,  dual  task  performance  could  not  benefit  from  automation  as  it  was  not  affected  by 
differences  in  compensatory  tracking  demands. 

Effects  of  all  independent  factors,  except  CHI,  were  reduced  with  increasing  update 
frequency.  It  seems  then  that  the  effects  of  CHI  and  update  frequency  are  additive.  In 
addition,  CHI  helped  independently  of  motor  load,  and  visual  load.  This  means  that  CHI 
helps  not  only  under  adversive  task  conditions  (as  far  as  the  three  manipulated  here),  but 
also  under  many  task  conditions  in  which  these  factors  are  more  optimal. 

Major  conclusion  of  the  present  study  is  that  CHI  at  a  control  device  helps  when  it  involves 
information  that  can  be  used  directly  for  vehicle  control.  In  addition,  when  it  provides 
information  that  is  less  relevant  for  control,  it  only  marginally  degrades  control  perfor¬ 
mance.  When  automation  partially  takes  over  control,  or  parts  of  the  control  process  are 
allocated  to  a  co-operator,  the  camera  operator  is  partly  separated  from  the  control  process 
and  may  loose  situation  awareness.  This  essential  aspect  of  teleoperation  may  be 
compensated  for  by  providing  additional  information  by  the  control  device  that  does  not 
significantly  interfere  with  the  primary  control  task.  In  that  case  the  operator  will  keep 
complete  control  over  the  system  while  being  able  to  cope  with  potential  orientation 
problems. 

One  note  has  to  be  made  in  connection  to  the  present  simulator  study.  As  a  consequence  of  a 
deliberate  choice  for  a  two  dimensional  presentation  a  number  of  (complicating)  factors  was 
left  out  of  consideration.  Therefore  it  might  be  that  this  conclusion  is  not  fully  applicable  to 
a  3D  situation.  The  results  of  a  study  by  Korteling  and  Van  der  Borg  (1994,  1997)  permit 
for  good  prospects.  Their  conclusion  that  semi-automation  in  combination  with  an  active 
interface  lead  to  improved  tracking  performance  compared  to  a  non-aided  platform  was 
based  on  experimental  results  from  a  3D  tracking  task. 

CHI  has  been  shown  to  be  useful  under  conditions  of  low  image  quality  and  both  high  visual 
and  motor  taskload.  Therefore  it  offers  a  promising  opportunity  in  teleoperation.  In  sofar  as 
high  taskload  is  concerned,  partial  task  automation  may  be  a  good  option  to  reduce  process¬ 
ing  demands.  CHI  may  in  this  case  be  used  to  maintain  a  sufficient  level  of  situation 


awareness. 
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APPENDIX  A  Technical  specifications  of  the  TNO-TM  teleoperation  simulator 


Display  monitor 

Manufacturer:  Mitsubishi,  type  Diamond  Pro  20; 

Principle:  CRT  video  colour  monitor,  multisync; 

Video  input:  standard  video  RGB-S; 

line  frequency  up  to  72  kHz; 
raster  frequency  up  to  100  Hz. 


Display  computer 


Manufacturer: 

Principle: 

Resolution: 

Colour: 

Video  output: 


Silicon  Graphics,  type  IRIS  4D; 

3D-images  of  graphical  processor,  colour; 

1280  x  1024  pixels  at  15  Hz; 

max  1024  colours; 

standard  video  RGB-S 

line  frequency  up  to  72  kHz. 


Joystick 


Manufacturer: 

Principle: 

Handling  force: 
Degrees  of  freedom: 
Spring  characteristic: 
Zero: 

Adjustability: 


Measurement  Systems  Inc.,  type  462L  miniature; 
force  sensor  controls  servomechanism  of  joystick; 
maximum  10  N  at  25mm  stick  height; 
two  horizontal,  one  rotation  (optional); 
programmable; 

programmable  (between  —30  and  30°); 
from  —30  to  30°. 


Computer  systems  for  supervision  en  position  calculation 


Manufacturer: 

Principle: 

Processors: 

Data  registration: 


standard  IBM  compatible  PC; 

network  of  processors  for  supervision  and  position  calculation; 
MSDOS  system  software  version  5,  clock  frequency  at  least  12  MHz; 
on-line  registration  real-time  sampling,  maximum  30  Hz. 
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APPENDIX  B  Instruction  for  subjects  (in  Dutch) 


Instructie  trackingexperiment 

Je  bevindt  je  straks  in  een  controlekamer  met  een  monitor,  joystick  en  een  aantal  knoppen. 
Op  de  monitor  zie  je  het  camerabeeld  van  een  camera  die  hoog  in  de  lucht  onder  een 
bewegend  platform  hangt.  De  camera  is  loodrecht  naar  beneden  gericht  op  een  gebied 
waarin  een  voertuigje  rijdt.  Het  is  de  bedoeling  dat  je  het  voertuig  zo  precies  mogelijk  in  het 
midden  van  het  camerabeeld  houdt.  Dit  doe  je  door  met  de  joystick  het  platform  van  koers 
te  laten  veranderen  zodat  de  camera  recht  boven  het  voertuigje  blijft. 

Als  er  twee  minuten  voorbij  zijn  wordt  de  missie  beeindigd  en  start  de  proefleider  de 
volgende  missie. 

Op  de  monitor  zie  je  verder  nog  acht  blokjes  staan  die  tezamen  een  kring  vormen.  Tijdens 
sommige  missies  kunnen  twee  van  deze  blokjes  af  en  toe  korte  tijd  oplichten  wanneer  je  de 
camera  aan  het  besturen  bent. 

Als  je  dit  ziet  gebeuren  tel  je  snel  het  aantal  blokjes  tussen  de  twee  oplichtende  blokjes 
(probeer  dit  te  doen  zonder  je  ogen  op  de  blokjes  te  richten).  Zitten  er  twee  blokjes  tussen, 
dan  druk  je  binnen  anderhalve  seconde  op  de  groene  knop.  Als  je  te  laat  bent  hoor  je  een 
vervelende  pieptoon  die  net  zolang  duurt  tot  je  alsnog  op  de  groene  knop  drukt.  Zit  er  maar 
een  of  zitten  er  drie  blokjes  tussen  reageer  je  niet!  Druk  je  toch  op  de  knop,  dan  hoor  je  een 
korte  dubbele  pieptoon. 

Elke  pieptoon  (of  dubbele  pieptoon)  telt  als  een  fout.  Probeer  absoluut  geen  fouten  te 
maken!!!! 

Je  krijgt  zo  dadelijk  eerst  even  de  gelegenheid  om  te  oefenen.  Mocht  je  vragen  hebben  kun 
je  die  altijd  aan  de  proefleider  stellen. 
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